RMAF: Relu-Memristor-Like Activation Function for Deep Learning
نویسندگان
چکیده
منابع مشابه
Nonparametric regression using deep neural networks with ReLU activation function
Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to log n-factors) under a general composition assumption on the regression function. The framework includes many well-studied structural constrain...
متن کاملDeep Learning using Rectified Linear Units (ReLU)
We introduce the use of rectified linear units (ReLU) as the classification function in a deep neural network (DNN). Conventionally, ReLU is used as an activation function in DNNs, with Softmax function as their classification function. However, there have been several studies on using a classification function other than Softmax, and this study is an addition to those. We accomplish this by ta...
متن کاملFunction approximation with ReLU-like zonal function networks
A zonal function (ZF) network on the q dimensional sphere S is a network of the form x 7→ ∑n k=1 akφ(x ·xk) where φ : [−1, 1]→ R is the activation function, xk ∈ S are the centers, and ak ∈ R. While the approximation properties of such networks are well studied in the context of positive definite activation functions, recent interest in deep and shallow networks motivate the study of activation...
متن کاملA Plausible Memristor Implementation of Deep Learning Neural Networks
A possible method for hardware implementation of multilayer neural networks with the back-propagation learning algorithm employing memristor cross-bar matrices for weight storage is modeled. The proposed approach offers an efficient way to perform both learning and recognition operations. The solution of several arising problems, such as the representation and multiplication of signals as well ...
متن کاملMemristor Models for Machine Learning
In the quest for alternatives to traditional complementary metal-oxide-semiconductor, it is being suggested that digital computing efficiency and power can be improved by matching the precision to the application. Many applications do not need the high precision that is being used today. In particular, large gains in area and power efficiency could be achieved by dedicated analog realizations o...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Access
سال: 2020
ISSN: 2169-3536
DOI: 10.1109/access.2020.2987829